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INTRODUCTION 


The  Advanced  Teleoperator  Technology  program  is  a  NAVMAT-funded  6.2  effort 
with  two  major  objectives.  The  first  objective  is  to  integrate  and  advance  the  technologies 
relevant  to  the  development  of  advanced  general  teleoperator  systems,  and  secondly,  to 
demonstrate  the  technical  feasibility  and  advantages  of  these  advanced  systems  to  perform 
a  variety  of  important  Navy  and  Marine  Corps  warfare  and  support  missions. 

This  program  is  based  on  the  concept  that  a  remote  system  which  capitalizes  on  the 
extensive  capabilities  and  adaptability  of  man  needs  to  be  investigated.  Man  is  incorporated 
into  the  system’s  control  loops  because  his  unique  ability  to  reason  and  adapt  to  changing 
conditions  cannot  be  replaced  with  near-term  available  computer  systems  (i.e.,  robots). 

A  part  of  this  program  will  develop  the  technology  necessary  to  supply  an  operator 
with  remotely-obtained  direct  and  extended  sensory  information  which  will  enable  the  op¬ 
erator  to  project  himself  into  remote,  and  possibly  hostile,  environments.  Situations  where 
such  a  system  might  provide  the  operator  with  a  safe,  on-the-scene  presence  are  diving  oper¬ 
ations;  emergency  rescue  and  repair;  handling  of  nuclear,  biological,  and  chemical  materials; 
combat  operations  in  support  of  amphibious  or  tactical  warfare  ashore;  and  space  operations. 

It  is  our  intention  to  begin  investigation  of  those  subsystems  that  will  acquire,  trans¬ 
mit,  and  “display”  direct  and  extended  sensory  information  to  the  operator  in  order  that  he 
“feel”  presence  at  the  remote  site. 

To  initiate  these  investigations,  the  Advanced  Teleoperator  Technology  Conference/ 
Workshop  was  conducted  October  3-4,  1979,  at  the  Naval  Post-Graduate  School,  Monterey, 
California.  The  conference,  hosted  by  the  Naval  Ocean  Systems  Center  (Hawaii  Laboratory), 
was  attended  by  leading  researchers  from  universities,  research  institutes,  program  sponsors, 
and  other  Navy  representatives  with  expertise  in  technology-related  areas.  A  complete  list  of 
participants  and  their  areas  of  expertise  follows  this  introduction.  Several  of  the  participants 
are  involved  in  development  of  aids  for  the  handicapped;  others  are  experts  on  different  sen¬ 
sory  modalities  (i.e.,  vision,  audition,  olfaction). 

The  two-day  conference  began  with  a  statement  of  the  program  background,  history, 
and  objectives,  followed  by  a  series  of  presentations  by  the  invited  sensory  experts  which 
included  brief  discussions  of  the  present  state  of  the  art  within  their  fields,  and  their  past 
and  present  work.  The  conference  concluded  with  an  afternoon  devoted  to  open  forum 
discussions. 

The  following  conference  proceedings  are  presented  in  three  parts.  First,  the  opening 
remarks  by  the  sponsor  and  program  management  are  summarized.  Second,  brief  overviews 
of  each  of  the  presentation  made  by  the  individual  participants  are  reported.  Formal  papers 
and  visual  aids  have  not  been  included  in  these  proceedings,  so  it  is  left  to  the  reader  to  con¬ 
tact  the  individual  participants  for  further  information.  Finally,  the  open  forum  discussions 
are  summarized  according  to  the  topics  discussed. 
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LIST  OF  PARTICIPANTS 


Dr.  Carter  C.  Collins,  Associate  Director 
Smith-Kettlewell  Institute  of  Visual  Science 
2200  Webster  Street 
San  Francisco,  CA  94115 

Vision  substitution  via  tactile  image  projection. 

Electrocutaneous  perception. 

Sensory  substitution. 

Dr.  John  W.  Hill 

Senior  Research  Engineer 

Artificial  Intelligence  Center 

Computer  Science  &  Technology  Division 

S.  R.  I.  International 

333  Ravenswood  Avenue 

Menlo  Park,  CA  94025 

Control  and  use  by  man  of  mechanical  manipulators;  robotics;  man-machine  systems 
and  human  information  processing.  Touch,  force  and  range  sensors  for  remote  manipulators. 

Robert  S.  Gales 
Supervision  Physicist 

Head,  Airborne  Acoustics  Branch,  Code  5121 
Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

Auditory  perception  of  acoustic  signals,  human  factors  in  electronics,  underwater  sound. 

Dr.  John  Lyman 

Head,  Biotechnology  Laboratory 
University  of  California  at  Los  Angeles 
Engineering  I ,  Room  3116 
Los  Angeles,  CA  90024 

Biomedical  engineering,  prosthetics  and  human  factors. 

Dr.  John  Silva 

Head,  Command  Support  Division,  Code  823 
Naval  Ocean  Systems  Center 
San  Diego,  CA  92152 

Human  factors  and  bioengineering. 

Dr.  David  L.  Blank 
Naval  Research  Laboratory 
Ocean  Sciences  Division 
Washington,  D.  C.  20375 


Olfaction  and  taste. 
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Dr.  Frank  A.  Saunders,  Senior  Scientist 
Smith-Kettlewell  Institute  of  Visual  Science 
2200  Webster  Street 
San  Francisco,  CA  94115 

One-  and  Two-Dimensional  Electrocutaneous  Tactile  Displays. 

Dr.  C.  Scott  Johnson 

Biophysics  Staff  Scientist,  Code  5102 

Department  of  Biosciences 

Naval  Ocean  Systems  Center 

San  Diego,  CA  92152 

Bio-acoustics,  bio-sonar,  and  electro-sensing. 

Dr.  Larry  J.  Leifer 
Mechanical  Engineering  Department 
Stanford  University 
Stanford,  CA  94305 

Manipulators  and  prosthetics. 

Dr.  Ross  L.  Pepper 

Research  Psychologist,  Code  5 1 2 

Naval  Ocean  Systems  Center 

Hawaii  Laboratory 

P.  O.  Box  997 

Kailua.  Hawaii  96734 

Vision  and  visual  perception. 

Dr.  Thomas  B.  Sheridan 
Department  of  Mechanical  Engineering 
Massachusetts  Institute  of  Technology 
Cambridge.  MA  02139 


Teleoperators,  supervisory  control. 


NOSC  HAWAII  LABORATORY  PERSONNEL 

John  D.  Hightower 

Head,  Environmental  Sciences  Department,  Code  S3 
Donald  K.  Moore 

Program  Manager,  Undersea  Systems,  Code  5302 
James  K.  Katayama 

Head,  Ocean  Systems  Division,  Code  533 
David  C.  Smith 

Electronics  Engineer,  Code  533 1 

Frank  P.  Armogida 
Mechanical  Engineer,  Code  5334 


INVITED  REPRESENTATIVES 

Dr.  Richard  T.  Swim,  Naval  Research  Laboratory 
Dr.  Gerald  S.  Malecki,  Office  of  Naval  Research 

Dr.  Yee-Yeen  Chu,  Perceptronics,  Representative  for  Office  of  Cybernetics  Technology, 
Defense  Advanced  Research  Projects  Agency 


SPONSORS 

Glenn  R.  Spalding.  Naval  Material  Command 
Lt.  Col.  Ray  W.  Bowles,  U.S.M.C. 


DOCUMENTATION 


Ronald  H.  Schneider 
Integrated  Sciences  Corporation 
Santa  Monica,  CA  90401 

Document  conference. 
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OPENING  REMARKS 


David  C.  Smith  (NOSC,  Hawaii)  -  Introduction  of  Participants  and  Conference  Purpose 

The  participants  in  the  conference  were  introduced  and  their  organizational  affilia¬ 
tions  and  areas  of  expertise  were  mentioned  (see  preceding  list).  It  was  explained  that  to 
initiate  the  first  year’s  program  effort,  the  conference  was  called  to  have  sensory  experts 
interact  with  Navy  scientists  and  engineers.  Participants  were  instructed  to  discuss  and  con¬ 
ceptualize  approaches  for  acquiring  and  “displaying”  sensory  information  to  system  opera¬ 
tors  so  that  they  feel  presence  at  a  remote  site. 

Information  sent  to  the  attendees  prior  to  the  conference  was  discussed,  and  each 
invited  sensory  expert  was  asked  to  include  in  his  presentation  a  technology  forecast  from 
within  his  area  of  expertise  for  the  next  5-  to  1 5-year  period.  “How  to  create  realism”  was 
explained  to  be  the  overall  theme. 


Glenn  R.  Spalding  —  Sponsor  Opening  Remarks 

Mr.  Spalding,  from  NAVMAT,  and  the  prime  sponsor  of  the  program,  stated  that  the 
program  was  in  the  early  stages  of  exploratory  development  He  explained  that  the  confer¬ 
ence  and  program  should  attempt  to  assess  the  technologies  relevant  to  advanced  teleopera¬ 
tors  and  their  application  to  Navy  and  Marine  Corps  needs.  He  said  the  program  should  nield 
the  technologies  with  the  applications. 


James  K.  Katayama  (NOSC,  Hawaii)  -  Program  Goals  and  Approach 

The  need  for  teleoperators,  and  the  objectives  and  approach  to  the  program,  were 
presented. 

Mr.  Katayama  explained  that  some  of  the  technologies  necessary  to  develop  new. 
advanced  teleoperators  already  are  available  or  are  moving  ahead  quite  rapidly.  The  areas 
that  are  progressing  well  are  those  pertaining  to  high  quality  data  links,  specifically  coaxial 
cables  and  fiber  optics.  Artificial  intelligence  is  developing,  but  it  is  unable  to  handle  unde¬ 
fined,  unpredictable  situations:  thus,  autonomous  teleoperators  (i.e..  robots)  are  not  prac¬ 
tical. 


In  order  to  build  the  new  generation  of  teleoperators  that  NOSC  envisions,  it  will  be 
necessary  to  make  man  an  integral  part  of  the  control  loop.  With  man  in  the  loop,  an  inter¬ 
active  system  can  be  built  that  will  enable  the  adaptability  of  man  to  be  exploited.  Mr. 
Katayama  explained  that  it  had  been  decided  to  emphasize  obtaining  and  displaying  high- 
quality,  remote  sensory  data  for  the  remote  system  operator. 

The  senses  of  vision,  hearing,  olfaction,  taste  and  touch  were  mentioned  as  topics  to 
be  discussed  during  this  conference.  He  also  expressed  his  interest  in  investigating  a  new 
generation  of  remote  manipulators  with  sensory  feedback. 
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The  Advanced  Teleoperator  Technology  program  intends  to  develop  the  necessary 
subsystem  building  blocks  that  could  be  immediately  incorporated  into  a  mission-specific 
system  and  which  might  eventually  be  integrated  into  a  more  generic  teleoperator  system. 
Specific  applications  will  be  examined,  and  the  subsystem  building  blocks  will  be  tailored  to 
those  applications  utilizing  money  from  sources  other  than  the  6.2  program.  It  was  empha¬ 
sized  that  the  program  should  not  focus  strictly  on  underwater  applications. 
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FORMAL  PRESENTATIONS 


Dr.  Ross  L.  Pepper  -  Vision 

Dr.  Pepper  reviewed  the  literature,  theories  and  models  of  vision  and  summarized 
ongoing  visual  perception  research  as  it  relates  to  future  teleoperator  systems.  The  review 
covered  topics  ranging  from  the  neurophysiology  of  the  eye  to  the  ONR-funded  work  on 
display  devices. 

Work  done  on  stereoscopic  displays  at  NOSC’s  Hawaii  Laboratory  was  described. 

Dr.  Pepper’s  efforts  have  been  directed  toward  exploring  the  differences  in  remotely-viewed 
manipulator  task  performance  while  using  stereoscopic  or  monocular  display  systems.  His 
research  has  shown  that  stereo  provides  a  performance  advantage  under  different  levels  of 
reduced  visibility,  but  that  the  magnitude  of  the  advantage  is  task-dependent. 

He  explained  that  research  has  shown  that  we  do  not  see  the  same  magnitude  of 
differences  using  TV  displays  to  test  mono-stereo  performance  as  we  see  when  mono-bio 
performance  is  assessed  under  directly-viewed  conditions.  Dr.  Pepper  feels  there  are  many 
reasons  which  might  account  for  some  of  the  differences,  including  unfaithful  reproduction 
of: 


•  Absolute  size  and  distance 

•  Absolute  and  relative  depth  cues 

•  Color  and  resolution 

•  Convergence  and  accommodation 

•  Motion  parallax 

Dr.  Pepper  has  proposed  research  that  would  enable  him  to  address  aspects  of  these 
issues  to  determine  the  relative  performance  significance  of  these  and  other  visual  and  per¬ 
ceptual  factors  for  remotely-controlled  manipulator  tasks,  and  for  vehicle  navigation  and 
guidance. 


Dr.  Frank  A.  Saunders  -  Sensory  substitution  tactile  displays 

Dr.  Saunders  explained  his  present  work  using  tactile  displays  for  the  presentation  of 
sound  to  deaf  children,  and  discussed  the  problems  which  occur  when  the  skin  is  stimulated 
electrically  to  achieve  a  tactile  sensation.  The  system  used  with  the  deaf  was  demonstrated, 
and  consists  of  32  individual  tactile  stimulators  worn  around  the  waist  on  a  belt.  A  bi-phase 
stimulating  pulse  results  in  zero  direct  current,  thus  avoiding  electroplating  the  skin  or  any 
other  effects  which  could  be  harmful. 

He  went  on  to  mention  that  tactile  stimulation  is  excellent  for  changing  dynamic 
inputs  because  constant  level  stimulation  habituates  the  tactile  sensory  channel.  Data  ob¬ 
tained  from  two-point  threshold  experiments  has  confirmed  this  because  performance  has 
been  shown  to  be  much  higher  when  the  points  are  presented  sequentially. 
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Dr.  Saunders  has  been  working  with  ten  deaf  children  in  the  Marin  County  schools, 
using  the  belt  in  conjunction  with  lip  reading,  and  has  been  getting  favorable  results.  The 
performance  of  his  subjects  has  been  equivalent  to  progress  in  learning  a  foreign  language. 


Dr.  John  Lyman  -  Prosthetics  research 

Dr.  Lyman  briefly  discussed  his  many  years'  involvement  in  prosthetics  research  and 
gave  insights  on  how  the  field  has  progressed  through  the  years.  Today,  he  is  working  with 
pattern  recognition  on  the  myoelectric  signals  from  the  surface  of  the  shoulder  to  control  an 
artificial  arm.  He  went  on  to  explain  that  they  have  not  been  very  successful  in  providing 
feedback  to  the  patient  using  electrocutaneous  mapping.  He  discussed  some  of  the  problems 
related  to  electrocutaneous  stimulation,  including  limited  dynamic  range  and  uneven  current 
densities  under  the  stimulating  electrodes. 

Dr.  Lyman  presented  a  preliminary  report  by  one  of  his  graduate  students  who  has 
managed  successfully  to  deliver  80  words  of  vocabulary  through  the  skin  at  speech  rates. 
These  studies  have  been  performed  using  three  different,  highly-motivated  subjects.  He  then 
explained  the  research  UCLA  has  done  on  electrocutaneous  stimulation  and  discussed  all  the 
variables  they  have  explored  during  their  two-point  threshold  experiments. 

The  importance  of  artificial  intelligence  applied  to  automatic  control  also  was 
discussed. 


Dr.  John  Silva  -  Biomedical  engineering  and  human  factors 

Dr.  Silva  discussed  his  extensive  work  in  biomedical  engineering  and  human  factors. 
He  explained  the  use  of  evoked  cortical  responses  for  measurement  of  visual  and  auditory 
system  performance  and  recent  developments  in  nonevasive  measurement  of  blood  pressure, 
echo  cardiography  and  computer-aided  cardiology. 

The  development  of  a  remote  communications  system  to  provide  expert  advice  to 
corpsmen  on  small  ships,  and  a  medical  support  stretcher  for  keeping  the  wounded  stabilized 
during  transport  were  presented.  His  work  in  the  development  of  a  head-rest  control  for  a 
wheel  chair  and  a  stand-up  wheel  chair  also  were  presented. 

In  assessing  the  state  of  the  art  in  biomedical  and  human  engineering.  Dr.  Silva  stated 
that  the  most  influential  factors  are  the  computer  and  the  microprocessor.  He  said  tremen¬ 
dous  inroads  are  being  made  in  the  brain  and  neurophysiology  areas  and  that  bio-chemical 
technology  also  is  on  the  verge  of  major  advances.  Klectro-optic  examinations  are  being 
made  routinely  inside  the  body,  and  he  believes  that  computer-aided  decision  making  and 
advanced  decision  theory  will  have  a  great  impact  on  advanced  remote  systems. 
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Robert  S.  Gales  -  Audition 


The  characteristics  of  the  human  auditory  system  were  described.  Mr.  Gales  discussed 
those  functions  of  the  ear  which  should  be  incorporated  into  a  remote  listening  system. 

•  Detection  of  emitting  targets 

•  Classification/recognition/identification 

•  Localization 

He  explained  that  the  ear  does  an  excellent  job  in  all  the  above  functions  because  it  is  backed 
up  with  a  large  memory  bank  that  enables  it  to  detect,  identify  and  localize  instantly. 

The  way  in  which  we  localize  sounds  in  space  was  covered  in  depth,  with  time  differ¬ 
ence  of  arrival,  intensity  and  phase  being  the  cues  we  use.  He  said  recent  research  indicates 
that  above  5000  Hz  we  localize  in  the  vertical  plane  by  analyzing  the  sound  reflections  off 
the  pinna  (the  outer  ear). 

The  presentation  was  concluded  with  a  description  of  the  phenomenon  known  as 
time  separation  of  pitch,  which  is  used  by  the  blind  to  determine  distance  from  large  objects. 


Dr.  David  L.  Blank  -  Olfaction 

The  interest  of  the  Naval  Research  Laboratory  in  understanding  the  neurophysiology 
of  smell  was  discussed.  Dr.  Blank  explained  that  what  most  of  us  think  of  as  smell  is  really 
referred  to  as  flavor  and  is  comprised  of  smell,  taste  and  other  inputs  from  various  nerve 
centers.  Smell  was  explained  to  be  less  important  in  man  than  in  most  other  animals,  and 
that  in  man.  smell  or  flavor  is  mainly  an  aesthetic  sense. 

Given  the  same  odorant  in  different  concentrations,  man  can  identify  only  four  or 
five  different  levels  of  concentration.  With  respect  to  absolute  identification  (naming  differ¬ 
ent  odors),  man  can  differentiate  only  about  16  different  odors  at  a  time.  Professional  per¬ 
fumers  may  differentiate  1 8  to  22  odors.  It  was  explained  that  this  sensory  system  is  slightly 
better  than  other  sensory  systems  in  respect  to  identification. 

Dr.  Blank  then  explained  the  neurophysiology  of  the  olfactory  system.  The  similar¬ 
ities  between  the  operation  of  a  gas  chromatograph  and  a  functioning  olfactory  system  were 
drawn,  and  the  differential  sensitivity  of  receptor  cells  to  various  odorants  was  discussed.  He 
said  that  each  individual  receptor  cell  is  sensitive  to  more  than  one  chemical,  so  a  different 
pattern  of  receptor  firings  is  associated  with  each  odorant  (i.e„  a  cross-fiber  pattern  code). 

He  said  an  artificial  nose,  using  a  gas  chromatograph,  could  be  sensitive  down  to 
10’,s  molar,  and  that  if  a  person  defined  what  chemicals  he  was  interested  in  looking  for.  a 
column  could  be  designed  especially  for  those  chemicals. 


Dr.  C.  Scott  Johnson  —  Biosonar  and  animal  electric  field  detection 


Dr.  Johnson  presented  a  short  motion  picture  of  an  experiment  with  a  diver  sonar 
which  used  transformed  (into  the  human  hearing  range)  porpoise  echolocating  signals.  It 
was  explained  that  humans  performed  as  well  as  porpoises  when  tasked  to  identify  a  stand¬ 
ard  target  when  presented  with  a  pair  of  targets. 

Dr.  Johnson  emphasized  the  importance  of  allowing  the  diver  full  use  of  his  head  to 
scan  the  targets,  and  pointed  out  that  an  observer  outside  the  pool  did  not  do  as  well.  He 
mentioned  that  the  discrimination  was  a  subjective  one.  and  that  the  task  came  quite  easily 
to  the  subject  with  very  little  learning  involved. 

Dr.  Johnson  went  on  to  describe  how  some  marine  animals  are  v«_ry  sensitive  to 
electric  fields.  Sharks  can  detect  an  electric  field  of  10"K  volts  per  cm,  or  one  volt  in  1000 
kilometers.  This  allows  them  to  navigate  using  the  earth's  magnetic  field.  He  stated  that  if 
we  could  simulate  the  shark’s  mechanism,  we  could  detect  the  bearing  to  a  submarine-size 
object  at  a  distance  of  10  kilometers.  He  observed  that  this  is  quite  remarkable  when  one 
remembers  that  the  background  noise  due  to  the  currents  causing  electric  field  gradients  in 
flic  oceans  is  1000  times  the  sensitivity  of  the  shark. 


Dr.  Carter  C.  Collins  -  Sensory  substitution 

Dr.  Collins  described  accomplishments  on  the  tactile  representation  of  pictorial 
information.  He  has  developed  a  device  which  uses  converted  TV  camera  images  to  drive  a 
two-dimensional  array  of  electrotactors  situated  on  the  user’s  abdomen.  Pulse  width  varia¬ 
tion  presents  the  “light’'  intensity  information.  A  blind  man  wearing  such  a  10  x  10  array 
learned  a  high-acuity-requirement  electronic  assembly  task  and  kept  up  with  sighted  workers 
on  an  assembly  line.  He  explained  that  after  ten  to  twelve  hours  of  practice  with  the  device, 
one  can  learn  to  recognize  a  few  dozen  objects.  Dr.  Collins  also  has  evaluated  the  device  as 
a  mobility  aid,  and  has  demonstrated  the  device  by  having  the  user  avoid  obstacles  while 
navigating  through  a  room. 

Also,  he  described  some  of  the  problems  of  presenting  the  electrical  stimulation  and 
keeping  the  skin  in  contact  with  the  electrotactors.  For  two-dimensional  displays,  mechan¬ 
ical  stimulation  was  recommended  for  comfort.  Jets  of  water  also  have  been  tried,  and  Dr. 
Collins  said  the  skin  does  not  do  well  in  handling  parallel  information,  but  added  that  good 
use  can  be  made  of  temporal  scanning. 

Dr.  Collins  described  work  done  on  his  own  in  developing  an  artificial  nose.  The 
“nose"  consists  of  three  pairs  of  individually-coated  thermistors  that  respond  differently  to 
the  given  odorants’  latent  heat  of  fusion.  The  three  absorbent  materials  used  were  lanolin, 
albumin  and  polyvinyl  chloride.  Air  carrying  the  odorant  is  pulled  across  the  thermistor  pairs 
and  the  normalized  ratio  of  amplitude  of  responses  from  the  three  materials  can  be  plotted 
and  then  classified. 


10 


Dr.  John  W.  Hill  -  Manipulators 


Dr.  Hill  explained  the  work  he  has  done  forS.R.I.  and  NASA  which  has  incorporated 
tactile  feedback  and  display  systems  into  remote  manipulators.  He  has  implemented  touch 
and  proximity  sensors  and  has  used  vibrotactors  and  air  jets  to  provide  touch  sensations  to 
the  user. 

He  showed  many  methods  of  both  contact  and  non-contact  type  sensors  using  passive 
and  active  means  to  achieve  range  and  proximity  information.  He  mentioned  that  all  were 
laboratory  systems,  and  to  date  no  one  has  developed  a  decent  practical  tactile  sensor. 

Dr.  Hill  made  a  strong  appeal  for  more  work  dedicated  to  developing  hands.  He  feels 
that  work  done  on  end  effectors  is  an  area  of  concern  that  has  not  progressed  in  the  last  ten 
years,  and  that  someone  should  be  working  on  two  fingers  and  an  opposing  thumb. 

Dr.  Hill  now  works  in  the  artificial  intelligence  group  at  S.R.I.,  and  says  there  is  an 
inexhaustible  number  of  excellent  ideas  that  need  to  have  money  and  good  engineering  put 
behind  them.  If  the  correct  thrust  can  be  made,  impressive  and  sophisticated  hardware  can 
be  developed  without  too  much  difficulty. 


Dr.  Larry  J.  Leifer  -  Manipulators 

Dr.  Leifer  explained  the  wide  array  of  disciplines  he  represents  at  Stanford  University, 
including  the  A. I..  L.L.,  M.E.,  and  robotics  groups.  His  background  is  in  biomedical  engineer¬ 
ing,  and  because  of  this  he  is  very  interested  in  using  manipulators  to  allow  quadriplegics  to 
do  things  for  themselves. 

He  is  oriented  strongly  toward  human  factors,  and  is  very  interested  in  the  man- 
machine  interface.  He  explained  his  heavy  reliance  on  the  use  of  distributed  microprocessors 
in  controlling  the  arm.  In  such  applications  he  has  incorporated  extensive  use  of  voice 
control. 


He  agreed  quite  emphatically  with  Dr.  Hill  that  extensive  work  needs  to  be  done  to 
develop  a  “cybernetic”  hand.  He  has  built  a  parallel-jawed  hand  that  was  developed  to  pick 
up  objects  standing  vertical  or  lying  horizontally.  In  his  present  work  he  is  trying  to  take  the 
details  of  the  task  away  from  the  user  and  be  able  to  control  the  manipulator  with  task 
commands. 


Dr.  Thomas  B.  Sheridan  -  Teleoperators  and  supervisory  control 

Dr.  Sheridan  stated  that,  with  the  introduction  of  computers  and  artificial  intelli¬ 
gence  into  the  control  of  teleoperators,  we  are  now  in  a  better  position  to  appreciate  what 
man  can  do,  and  understand  how  people  and  computers  can  work  together. 

For  a  long  time.  Dr.  Sheridan  has  been  interested  in  mixing  human  and  computer 
control.  He  explained  his  ideas  on  supervisory  control  of  manipulators  and  undersea  vehicles 
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He  presented  a  short  motion  picture  on  work  that  has  been  done  at  MIT  with  a 
supervisory-controlled  manipulator,  and  explained  present  studies  investigating  manipula¬ 
tion  tasks  using  constrained  visual  systems.  The  present  studies  measure  operator  perform¬ 
ance  while  varying  the  presented  TV  picture  resolution,  frame  rate,  and  the  number  of  grey 
levels. 


The  question  was  raised  as  to  whether  or  not  it  is  desirable  to  build  a  hi-fi  sensing 
system  or  a  sophisticated  computer  controlling  system.  He  suggests  it  is  not  desirable  to  do 
either  because  they  should  be  made  to  complement  each  other.  He  wants  someone  to  build 
a  high-fidelity  sensor  system  that  will  make  it  easier  for  the  operator  to  program  the  com¬ 
puter  for  supervisory  control. 

Dr.  Sheridan  agrees  that  a  need  exists  for  better  sensing  equipment,  and  that  talk  of 
the  extensive  use  of  artificial  intelligence  is  like  talking  about  fusion  power.  "It  ain't  gonna 
be  here  for  a  long  time,”  he  said. 


Dr.  Richard  T.  Swini 

Dr.  Swim  was  invited  from  the  Naval  Research  Laboratory  to  explore  that  organiza¬ 
tion’s  potential  inputs  into  the  program. 

He  described  the  project  in  which  a  low-speed,  laminar-flow,  free-swimming  sub¬ 
mersible  was  developed.  The  vehicle  is  4  feet  in  diameter,  20  feet  long,  and  weighs  8,000 
pounds.  It  has  an  internal  volume  of  1 25  cubic  feet.  40  of  which  can  be  used  to  carry  cargo 
or  other  experiments.  The  submersible  runs  on  automobile  batteries  for  25  hours,  enabling 
it  to  travel  1 25  miles.  It  runs  under  microprocessor  programmed  control  and  has  sensors 
that  maintain  speed  of  advance  and  measure  roll,  pitch,  and  yaw.  Water  salinity  and  temper¬ 
ature  also  are  measured  and  recorded. 
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OPEN  FORUM  DISCUSSION 


FORM  OF  THE  SUMMARY 

The  open  forum  discussion  was  an  informal,  unstructured  dialogue  which  progressed 
freely  from  one  topic  to  another.  Therefore,  it  seemed  better  to  summarize  the  viewpoints 
by  subject  matter  rather  than  in  chronological  order.  The  open  forum  nature  of  the  confer¬ 
ence  enabled  many  different  points  of  view  on  teleoperator-related  subjeets  to  be  expressed. 
This  created  an  excellent,  thought-provoking  atmosphere.  The  discussions  dealt  primarily 
with  the  following  subjects: 

•  System  Characteristics 

•  Anthropomorphism 

•  Realism  or  Fidelity 

•  Operator  Involvement  in  Task  Performance 

•  Mission/Task  Characteristics 

•  Technology  State-of-the-art 

•  General  Topics 

•  Artificial  Intelligence 

•  Manipulators  and  Sensors 

•  System  Development 

SYSTEM  CHARACTERISTICS 
Anthropomorphism 

The  major  questions  addressed  in  this  area  were,  “How  much  should  the  teleoperator 
physically  resemble  man?",  and  “How  should  it  incorporate  human  performance  character¬ 
istics?”  Dr.  Sheridan  pointed  out  the  pros  and  cons  of  these  ideas. 

He  said  we  tend  to  make  things  in  our  own  image  when  we  do  not  understand  them 
well.  It  is  when  we  understand  more  about  the  functions  that  we  realize  we  can  perform  the 
tasks  better  with  systems  that  do  not  look  quite  so  human. 

In  opposition  to  this.  Dr.  Sheridan  said  that  if  the  teleopcrator  is  structured  like  man, 
it  is  much  easier  for  the  operator  to  spatially  identify  with  it.  and  this  is  necessary  if  stimulus 
incompatibilities  are  not  desired.  Stimulus  incompatibilities  must  be  kept  to  a  minimum  if 
the  operator  is  to  keep  track  of  the  position  of  the  teleoperator  “limbs."  A  system  with  pro¬ 
prioceptive  feedback  (feedback  from  sensors  in  the  limbs)  would  be  required. 

Perhaps  the  consensus  was  expressed  best  by  Dr.  Leifer.  He  stated  that  what  we  want 
is  to  perceive  the  teleoperator’s  sensors  as  our  own.  and  in  all  respects,  the  display  and  con¬ 
trolling  hardware  should  be  transparent  to  the  operator. 


Realism  or  Fidelity 


Mr.  Hightower  feels  strongly  that  a  need  exists  for  a  real-time,  high-fidelity  system 
between  the  remote  environment  and  the  human  operator.  He  pointed  out  that  many  of  the 
conference  participants  are  helping  the  handicapped  who  are  deprived  of  some  sensory  or 
motor  capabilities,  and  that  the  operator  of  the  teleoperator  will  be  chosen  because  he  has 
full  or  exceptional  capabilities. 

Mr.  Schneider  pointed  out  that  in  many  cases  the  hostile  environments  deprive  the 
system  of  fidelity  and,  in  some  cases,  all  use  of  a  particular  sensory  modality.  For  example, 
turbid  water,  smoke,  fire  or  darkness  severely  deprive  us  of  the  use  of  the  visual  sense.  The 
concept  of  sensory  substitution  as  a  method  of  solving  these  types  of  problems  was  discussed 
at  length.  Sensory  extension,  such  as  extended  vision  to  the  1R  or  UV  wavelengths,  or  ex¬ 
tending  audition  to  the  ultrasonic  or  infrasonic  frequencies,  were  among  the  topics  discussed. 

The  discussion  pursued  realism  and  its  importance  when  dealing  with  the  number  of 
degrees  of  freedom  needed  for  remote  manipulation.  Today's  manipulators  have  far  fewer 
degrees  of  freedom  than  the  human  hand,  and  provide  little,  if  any.  feedback  to  the  operator. 
Generally,  it  was  agreed  that  while  it  may  not  be  necessary  to  provide  all  the  degrees  of  free¬ 
dom  of  the  human  hand  and  arm,  more  than  the  three  to  seven  degrees  currently  available 
on  today’s  manipulators  would  be  useful,  and  that  more  realism  should  be  incorporated  into 
the  implementation.  The  importance  of  providing  the  operator  with  tactile  and  kinesthetic- 
feedback  also  was  stressed. 

Realism  in  terms  of  the  visual  sense  was  discussed.  A  system  incorporating  color, 
foveal-peripheral  relationships,  head-  and  eye-movement-following,  variable  convergence  and 
accommodation  were  envisioned.  It  was  stated  that  head-coupled  TV’s  have  been  around  for 
years  and  that  a  system  had  shown  its  utility'.as  part  of  the  Remote  Unmanned  Work  System 
in  Hawaii.  It  was  in  development  and  eventually  removed  from  the  system  because  of  budget 
limitations. 

Also  mentioned  was  a  head-coupled,  helmet-mounted  stereo  system  that  would  en¬ 
hance  realism  by  removing  the  inside-looking-out  (through  the  window)  effect  associated 
with  the  most  of  today’s  viewing  systems. 

Dr.  Blank  and  Dr.  Saunders  suggested  that  improved  sensors  could  be  built  to  take 
advantage  of  the  principle  of  lateral  inhibition  in  order  to  improve  edge  detection. 


Operator  Involvement  in  Task  Performance 

These  discussions  focused  on  the  degree  to  which  the  operator  should  be  involved  in 
the  details  of  a  task.  Dr.  Leifer  proposed  considering  the  following  metric: 

Atomic  Level  - - -  Complex 

(Simple  motor  tasks)  Tasks 
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He  suggested  that  man  really  wants  to  work  at  the  level  of  directing  tasks,  not  the  atomic 
level  of  guiding  each  individual  movement.  Some  of  the  other  participants  disagreed  with 
that  viewpoint,  saying  that  man  wants  to  be  more  involved  and  feel  that  he  has  things  more 
under  his  control. 

Mr.  Katayama  and  Dr.  Collins  expressed  the  view  that  if  all  of  the  senses  and  motor 
capabilities  can  be  provided,  microprocessors  and  programming  can  be  added  to  the  system 
to  do  the  more  mundane  portions  of  the  job  automatically. 


MISSION /TASK  CHARACTERISTICS 

A  great  deal  of  the  discussion  was  concerned  with  the  need  to  define  the  missions 
and  specific  tasks  which  teleoperators  would  be  used  to  perform.  The  main  controversy  was 
over  the  need  for  specificity  in  the  definition  of  tasks.  Dr.  Leifer  suggested  the  need  for  a 
goal,  even  if  that  goal  were  fanciful,  so  that  accomplishments  could  be  assessed. 

Dr.  Malecki  said,  and  most  participants  agreed,  that  the  desired  goals  and  ultimate 
users  must  be  identified.  He  said  that  at  least  the  classes  of  tasks  need  to  be  defined.  Mr. 
Hightower  said  that  not  only  is  a  list  of  all  task  functions  needed,  but  priorities  must  be 
assigned.  Dr.  Silva  commented  that  a  human-sensory-based  technological  platform  was 
needed,  but  to  build  something  useful,  the  tasks  it  will  perform  must  be  more  specifically 
selected. 


Dr.  Silva  also  said  that  in  examining  the  tasks,  the  environment  in  which  the  tele¬ 
operator  is  placed  must  be  monitored  to  determine  what  information  is  relayed  to  the  oper¬ 
ator  and  hoc.  that  information  is  displayed. 

Lt.  Col.  Bowles  stated  that  a  big  military  problem  today  is  target  acquisition  (e  g.. 
Where  is  the  target  right  now?).  Mr.  Armogida  said  that  he  had  looked  at  some  missions  and 
tasks.  Ordnance-disarming,  diver-manipulating,  and  marine  reconnaissance  tasks  also  were 
discussed. 


TECHNOLOGY  STATE-OF-THE-ART 
General 


NOSC  personnel  raised  the  question  as  to  whether  or  not  technology  can  support 
the  development  of  the  concept  of  projecting  the  feeling  of  man's  presence  into  the  tele¬ 
operator's  environment.  They  wanted  to  know  where  technology  stands  today,  and  solicited 
new  ideas  on  approaches  to  solving  this  problem. 

Dr.  Lyman  stated,  and  others  supported,  the  need  for  a  state-of-the-art  survey. 

From  the  sponsor's  point  of  view,  Lt.  Col.  Bowles  said  lie  wanted  to  know  where  the 
technological  risks  are  in  initiating  this  type  of  program,  and  what  the  opposition  (mainly 
USSR)  is  doing  in  this  area.  Dr.  Lyman  responded  to  the  latter  question  by  saying  the  Soviets 
probably  are  well  behind  us  at  this  time. 
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Several  participants  expressed  the  thought  that  today’s  technology  can  probably 
support  the  program  but  that  the  major  problem  is  finding  the  money  that  would  support 
integrating  all  those  pieces. 


Artificial  Intelligence  (AI) 

Dr.  Lyman  was  asked  to  comment  on  artificial  intelligence.  He  replied  that  AI  ad¬ 
vances  are  being  aided  by: 

•  Cheap  memory  (good  concepts  require  dense  memory). 

•  Ability  to  manipulate  data  in  condensed  form  (e.g.,  use  of  polynomial 
functions  which  describe  a  large  amount  of  data). 

Dr.  Sheridan  stated  that  parallel  processing  permits  much  more  processing  at  a  much  faster 
rate,  but  cautioned  that  AI  people  tend  to  be  unrealistic  in  terms  of  the  time  it  takes  to  get 
the  desired  results. 

Dr.  Hill  stated  that  years  ago,  artificial  intelligence  was  oversold,  but  now  results  are 
being  seen  in  the  areas  of  speech  and  image  recognition.  He  cautioned  not  to  expect  too 
much  work  that  will  be  directly  applicable  to  teleoperators. 


Manipulators  and  Sensors 

Dr.  Lyman  said  that  sensor  development  is  proceeding  at  a  rapid  pace,  particularly  in 
the  optics  area,  and  said  that  you  can  buy  eyes  (miniature  TV  systems)  but  have  problems 
with  visual  display.  References  were  made  to  the  development  of  eye-movement  trackers 
and  the  existence  of  a  tracker  that  does  not  require  fixing  the  head. 

The  conferees  agreed  that  the  tactile  skin  sense  presently  is  missing  and  that  it  is  a 
very  important  area  where  little  work  is  being  done.  Interest  was  expressed  in  a  “second  skin" 
being  built  by  Union  Carbide  for  prostheses. 

Problems  of  disorientation  due  to  the  operator  not  being  coupled  to  the  motion  of 
remote  undersea  vehicles  were  discussed.  If  the  water  suddenly  becomes  turbid  and  the  ve¬ 
hicle  moves,  the  operator  has  no  idea  where  he  is  in  relation  to  the  vehicle’s  original  position 
when  the  water  calms.  Mr.  Hightower  said  the  Soviet  literature  referred  to  similar  problems 
which  were  reduced  when  the  operator,  in  a  dentist-like  chair,  was  coupled  into  vehicle 
motion. 


Dr.  Lyman  explained  how  much  preprocessing  of  sensory  data  takes  place  before  it 
gets  to  the  brain,  and  the  question  was  raised  of  how  much  preprocessing  should  be  done  at 
the  remote  site  (i.e.,  before  it  is  sent  to  the  operator).  He  suggested  that  all  direct  sensory 
data  could  be  given  to  the  operator  in  the  “raw”  form  because  there  is  no  sensory  processor 
that  approaches  the  capability  of  the  brain. 
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While  it  was  pointed  out  by  Dr.  Hill  that  manipulators  can  be  made  stronger,  faster, 
and  more  accurate  than  the  human  hand  and  arm,  the  sensors  are  the  items  that  are  sorely 
lacking.  Dr.  Lyman  agreed,  and  said  that  the  sensory  capabilities  of  man  are  an  order  of 
magnitude  better  than  his  manipulative  capability. 

Mr.  Hightower  felt  the  highest  near-term  priorities  were: 

•  Building  a  hand  that  has  the  manipulative  capabilities  of  humans  (adaptable 
for  many  tasks). 

•  Recreating  the  visual  aspects  of  the  scene. 

He  restated  his  thoughts  on  the  importance  of  the  adaptability  of  man,  and  that  the  system 
should  take  advantage  of  what  man  has  to  offer. 

Mr.  Smith  said  that  NOSC  has  obtained  a  visual  tracking  system  from  Honeywell 
which  is  used  in  an  operational  airborne  missile  guidance  system.  Mr.  Katayama  said  that 
in  a  year  or  a  year  and  a  half  NOSC  would  have  a  head  (with  eyes  and  ear  on  an  articulated 
boom,  but  he  does  not  think  the  eye  will  have  the  foveal-peripheral  relations  of  the  human 
eye. 

Dr.  Johnson  expressed  the  opinion  that  visual  displays  should  not  be  substituted  for 
the  auditory  channel  in  presenting  the  operator  with  acoustic  information.  Mr.  Gales  sup¬ 
ported  that  viewpoint. 

The  problem  of  locomotion  was  touched  on  only  briefly,  and  Mr.  Smith  said  the 
program  did  not  expect  to  build  a  teleoperator  that  walked  like  a  man. 


SYSTEM  DEVELOPMENT 

Three  general  items  of  importance  were  expressed: 

•  The  need  to  investigate  both  the  technologies  and  missions  and  to  attempt 
to  interrelate  the  two  -  (stressed  by  Mr.  Spalding).  Dr.  Malecki  thought 
this  should  be  done  in  a  quantitative,  integrated  way. 

•  The  importance  of  investigating  ( 1 )  the  teleoperator  as  a  system  and  defining 
the  specific  components,  and  (2)  different  types  of  operator  displays  and 
controls  (a  point  made  by  Dr.  Silva). 

•  The  need  to  develop  an  approach  that  will  pull  advancing  technologies 
together  and  integrate  them  into  a  teleoperator  system  (stated  by  Dr.  Pepper 
and  others).  Dr.  Pepper  believes  there  will  be  no  problem  in  marketing  an 
integrated  system  once  development  has  been  initiated. 


RECOMMENDATIONS 


From  the  open  forum  discussions,  the  following  items  were  extracted  as  recommen 
dations  for  the  Advanced  Teleoperator  Technology  program.  Some  of  these  items  already 
are  in  the  preliminary  program  plan.  It  may  be  beyond  the  resources  of  the  program  to  pur 
sue  all  the  recommendations,  but  other  funding  will  be  sought  and  other  programs  will  be 
encouraged  to  insure  that  all  important  aspects  of  the  technology  are  adequately  treated. 

•  Analyze  missions,  develop  task  descriptions,  and  prepare  scenarios  of 
potential  teleoperator  development. 

•  Conduct  a  state-of-the-art  survey  of  the  technologies  applicable  to  tele¬ 
operator  development. 

•  Compare  mission  needs  with  the  state  of  the  art  and  identify  gaps  and  risks. 

•  Set  goals.  Decide  what  kind  of  teleoperator  (if  any)  is  to  be  built  and  what 
its  characteristics  and  capabilities  will  be. 

•  Impart  to  the  operator  the  feeling  that  he  is  the  teleoperator  at  the  remote 
location.  Strive  for  high  fidelity  in  simulating  the  human  senses  at  the  tele¬ 
operator  and  present  the  information  to  the  operator  in  real  time. 

•  Improve  sensors  with  respect  to  incorporating  more  of  the  characteristics 
which  add  realism  (e.g.,  eye  movement,  foveal-peripheral  relations,  variable 
convergence,  etc.). 

•  Develop  new  sensors  for  olfaction,  proprioception  and  touch. 

•  Develop  a  manipulator  with  more  degrees  of  freedom  (greater  than  6)  and 
attempt  to  create  realistic  control  by  modeling  the  end  effector  like  the 
human  hand.  Incorporate  the  senses  of  touch  and  proprioception  in  that 
manipulator. 

•  Integrate  the  products  of  relating  technologies  (sensors,  manipulators, 
artificial  intelligence,  etc.). 
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